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Pyrotechnic  Device  Reliability 

Lyn  R.  Whitaker  and  Michael  P.  Bailey 

Department  of  Operations  Research 

Naval  Postgraduate  School 

Monterey,  CA  93943 

(408)646-3482 
December  19,  1991 

Abstract 

This  report  looks  at  the  problem  of  determining  when  to  award  bonuses  for  reliability 
improvement  in  pyrotechnic  devices  based  on  data  gathered  under  a  lot  acceptance 
sampling  plan. 

1      Introduction 

As  government  procurement  contracts  are  now  written,  contractors  have  no  incentive  to 
improve  the  quality  of  the  items  they  provide.  To  improve  quality,  the  Naval  Weapons 
Support  Center.  Crane,  IN,  has  decided  to  implement  a  bonus  system.  If,  as  a  result  of 
the  sampling  inspection  plan  currently  required  for  lot  acceptance,  there  is  evidence  that 
the  lot  has  a  reliability  that  exceeds  the  minimum  requirement  for  lot  acceptance,  then  the 
contractor  will  be  eligible  for  a  bonus.  The  difficulty  with  implementing  such  a  system  of 
bonuses  is  that  the  sampling  inspection  plans  are  designed  to  provide  clear  cut  criteria  for 

1 


lot  acceptance  or  rejections.    They  do  not  readily  provide  a  single  measure  of  quality  or 


reliability  of  the  item  being  tested. 

The  first  item  for  which  a  bonus  system  will  be  implemented  is  a  pyrotechnic  device. 
The  sampling  inspection  plan  for  this  device  involves  sampling  items  from  a  lot.  subjecting 
them  to  a  manufacturing  test  and  exposing  them  to  one  of  three  environments.  An  attempt 
is  then  made  to  activate  each  device.  Specifically,  a  lot  of  500  to  1000  devices  is  accepted  if 

1.  Of  20  items  subjected  to  the  manufacturing  test,  no  more  than  1  can  fail  to  activate. 

2.  Of  20  items  subjected  to  the  joint  manufacturing  and  temperature  and  humidity  test 
no  more  than  1  ran  fail  to  activate. 

3.  Of  32  items  subjected  to  the  joint  manufacturing  and  vibration  test  no  more  than  2 
can  fail  to  activate. 

4.  Of  20  items  subjected  to  the  joint  manufacturing  and  altitude  test  no  more  than  1 
ran  fail  to  activate. 

The  samples  in  tests  1-4  are  distinct  .  thus  a  total  of  92  items  are  tested. 

Attempts  have  been  made  to  estimate  the  reliability  R  of  a  device  (here  reliability  is 
defined  as  the  probability  that  an  item  would  activate  after  the  manufacturing  test  and 
exposure  to  all  three  environments)  based  on  data  from  the  sampling  inspection  plan  ([4] 
and  [o])  .  The  methods  used  to  produce  lower  confidence  bounds  I, OH  for  this  reliability 
based  on  data  from  a  lot  which  is  just  barely  accepted  have  yeilded  values  that  are  much 
lower  than  observed  in  the  field. 

The  most  obvious  reason  for  this  discrepancy  is  that  requiring  successful  activation  after 
exposure  to  all  environments  is  much  more  stringent  than  requiring  a  device  off  the  shelf 


to  activate  in  practice.  An  operational  environment  would  not  include  all  these  adverse 
conditions.  It  is  also  likely  that  items  which  activate  after  exposure  to  one  environment 
would  be  more  likely  to  activate  after  exposure  to  other  environments.  Unfortunately  the 
presence  or  absence  of  such  dependence  cannot  be  inferred  from  the  data  yielded  by  the 
current  sampling  inspection  plan.  Thus,  the  models  that  have  been  used  to  estimate  the 
reliability  have  assumed  independence  of  the  events  that,  a  device  activates  after  exposure  to 
the  three  environments  given  that  it  would  have  activated  after  the  manufacturing  test.  If 
there  is  positive  dependence  between  these  events  then  estimation  under  the  assumption  of 
independence  yields  reliabilities  and  LCB's  that  are  lower  than  they  would  be  if  dependence 
were  accounted  for.  It  should  be  noted  that  even  if  exact  LCB's  can  be  found,  if  the 
modeling  assumptions  are  changed  to  include  dependence  (where  these  new  assumptions 
are  supported  by  empirical  evidence),  and  if  the  true'  reliability  could  be  defined  and 
estimated  from  the  available  data,  the  LCB's  could  still  be  rather  low  for  the  simple  reason 
that  the  sample  sizes  of  the  sampling  inspection  plan  are  too  small  to  give  a  sharper  lower 
confidence  bound. 

In  Section  2,  we  find  the  maximum  liklihood  estimator  (MLE)  of  R  under  the  assump- 
tion of  independence  of  the  events  that  a  device  activates  after  exposure  to  the  three  en- 
vironments given  that  it  has  passed  the  manufacturing  test.  In  the  process,  we  uncover  a 
difficulty  with  estimating  R  under  this  assumption.  Specifically,  this  procedure  can  yield 
lower  estimates  of  R  for  samples  which  have  fewer  failures.  To  fix  this,  lower  bounds  for 
the  MLE  of  R  under  the  conditional  independence  assumption  are  derived  and  shown  to 
be  lower  bounds  for  MLE's  under  weaker  and  more  realistic  assumptions.  Approximate 
95%  LCBrs  for  R  and  the  MLE  arc  provided  explicitly  for  the  2i  realizations  of  the  sam- 
pling plan  leading  to  lot  acceptance,    hi  Section  3  a  direct  approach  to  awarding  bonuses 
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is  taken.    It  provides  an  alternative  to  using  LCB's  for.  decision  making.   Conclusions  and 
recommendations  are  given  in  Section  4. 


2      Estimating  Reliability 

Tests  2.  3.  and  4  in  the  lot  acceptance  sampling  plan  involve  trying  to  activate  the  device 
after  it  is  subjected  to  the  manufacturing  test  and  one  of  the  environmental  tests.  To 
distinguish  between  the  outcome  of  these  joint  tests  and  what  would  have  happened  had 
the  device  been  subjected  to  one  of  the  tests  alone,  let  E\,E<y,E%,  and  £4  be  the  respective 
events  that  a  device  can  activate  after  being  subjected  to  just  the  manufacturing  test,  the 
temperature  and  humidity  test,  the  vibration  test,  and  the  altitude  test.  Then.  R\  =  P{E\) 
and  R,  =  P(E\  D  £*,-),  i  =  2,3,4  are  the  probabilities  that  a  device  activates  after  being 
suhjeetpd  to  tests  1,  2.  3.  or  4.  respectively.  The  reliability  of  a  device  is  defined  as 

R  =  P(Ei  n  £"2  n  £3  n  £4). 

H  will  be  estimated  based  on  Xt.  the  number  of  devices  which  activate  out  of  the  nl 
that  are  subjected  to  test  i.  The  sampling  plan  uses  different  devices  randomly  chosen  from 

the  lot  in  question.  Thus,  it  is  reasonable  to  assume  that  A'] V}  are  independent  and 

Xt  --  Li inomial{nt.  R,).  i  =  1 4. 


2.1      Maximum  Likelihood  Estimation 

The  likelihood  of  observing  X\  =  x\ A4  =  £4  is  given  as 

,  / ..  \ 


'  =  n 


fff  (1  -  R,) 


rt,  —  J, 


(1) 


V  '•  / 


where  0  <  /?]  <  1,0  <  R, <  /?],/  =  2.3.-4.  For  the  sampling  plan  described  in  the  intro- 
duction, there  are  24  realizations  of  X{ . . .  ,  X{  that  lead  to  lot  acceptance.  The  maximum 
likelihood  estimates  (MLE's)  R{  of  Rt  for  these  24  realizations  are  given  in  Table  1. 

Because  the  same  device  is  never  actually  subjected  to  more  than  one  of  the  environmen- 
tal tests,  the  estimators  Rt  of  Rt  alone  are  not  enough  to  construct  an  MLE  for  R.  In  the 
subsequent  subsections,  we  describe  an  estimator  Rci  f°r  P  which  is  ba.sed  on  the  assump- 
tion of  conditional  independence.  After  examining  the  shortcomings  of  Rci-,  we  provide  a 
second  estimate  Rig  which  requires  no  assumptions  concerning  dependencies  between  test, 
outcomes,  and  which  provides  a  lower  bound  for  R. 

2.2     Conditional  Independence 

Additional  assumptions  about  the  dependence  between  the  events  E\ . . . ,  £4  are  needed 
to  estimate  R  based  on  the  current  sampling  plan  data.  The  most  direct  approach  for 
estimating  R  based  on  this  data  is  to  assume  that,  given  F.\.  that  En.  £3.  F.\  are  independent, 
events,  (see  [4]  and  [5]  for  further  discussion.)  The  MLE's  in  Table  1  are  also  the  MLE's  for 
Rr  under  the  assumption  of  conditional  independence.  With  this  assumption,  when  R\  >  0, 
we  have 

R   =   P(E1nE2nE3nEA) 

=     P{E2DE^nE4\Ei)P{Ei) 

=    P{E2\E1)P(E3\El)P(EA\E1)P(E1) 
R^R3R4 

Thus  the  MLE  Rci  of  R  under  this  assumption  is  given  by 


Table  1:  MLEs  for  R[,. . .  ,R^  computed  for  the  24  realizations  of  the  sampling  plan  leading 
to  lot  acceptance,  where  (i,j,  k,  I)  represents  the  number  of  failures  in  tests  1, . . . ,  4. 


W/fcfl  .. 

*l 

RZ 

*3 

*4 

(0000) 

1.000 

1.000 

1.000 

1.000 

(0001) 

1.000 

1.000 

1.000 

0.950 

(0010) 

1.000 

1.000 

0.969 

1.000 

(0100) 

1.000 

0.950 

1.000 

1.000 

(1000) 

0.989 

0.989 

0.989 

0.989 

(0011) 

1.000 

1.000 

0.969 

0.950 

(0101) 

1.000 

0.950 

1.000 

0.950 

(0110) 

1.000 

0.960 

0.969 

1.000 

(1001) 

0.986 

0.986 

0.986 

0.950 

(1010) 

0.983 

0.983 

0.969 

0.983 

(1100) 

0.986 

0.950 

0.986 

0.986 

(0020) 

1.000 

1.000 

0.938 

1.000 

(0111) 

1.000 

0.950 

0.969 

0.950 

(1011) 

0.975 

0.975 

0.969 

0.950 

(1101) 

0.981 

0.950 

0.981 

0.950 

(1110) 

0.973 

0.950 

0.969 

0.975 

(0021) 

1.000 

1.000 

0.938 

0.950 

(0121) 

1.000 

0.950 

0.938 

1.000 

(1020) 

0.983 

0.983 

0.938 

0.983 

(1111) 

0.962 

0.950 

0.961 

0.950 

(0121) 

1.000 

0.950 

0.938 

0.950 

(1021) 

0.975 

0.975 

0.938 

0.950 

(1120) 

0.975 

0.950 

0.938 

0.975 

(1121) 

0.950 

0.950 

0.938 

0.950 

Rci  =  < 


R'l 


0  III  =  o. 

Realizations  of  Rci  for  the  24  outcomes  that  lead  to  lot  acceptance  are  given  in  Table 
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From  Table  2  it  is  clear  that  the  assumption  of  conditional  independence  is  unacceptable. 
Under  conditional  indepence.  there  are  several  cases  where  sampling  plans  resulting  in  fewer 
failures  give  smaller  estimated  value  of  R  than  with  more  failures.  An  example  of  this 
is  the  failure  configuration  (1.1.1.0)  with  Rci  —  -044  compared  to  the  case  (0.1.1.0) 
with  Rci  —  -920.  What  happens  in  the  (0. 1.  1,0)  case  is  that  the  conditional  indepence 
assumption  will  attribute  a  disproportionate  amount  of  the  cause  of  failure  to  the  lot's 
inability  to  pass  environmental  tests  2  and  3.  Thus,  the  estimated  probability  of  passing  all 
tests  is  downgraded  too  much.  This  contradiction  will  lead  us  to  dispose  of  the  conditional 
independence  assumption  altogether.  We  will  now  make  the  results  of  this  phenomenon 
formal. 

Data  from  the  lot  acceptance  sampling  plan  can  be  thought  of  as  censored.  For  tests  2-4. 
it  is  unknown  whether  failure  is  due  to  the  manufacturing  test,  the  particular  environmental 
test,  or  both.  This  lack  of  information,  along  with  the  conditional  independence  assumption, 
causes  the  apparent  reversals  in  Rci  observed  in  Table  2.  To  see  this,  suppose  that  we  could 
determine  the  cause  of  failure  for  a  particular  device  which  fails  to  activate.  For  test  i.  let 

•  rii  —  x,  be  the  total  number  of  failures  for  test  i: 

•  Jii  be  the  number  of  failures  resulting  from  only  the  environmental  exposure  (these 
would  have  passed  the  simple  manufacturing  test); 

•  Xt.2  be  the  number  of  failures  resulting  from  only  the  manufacturing  test,  (these  would 


Table  2:  The  MLE  for  R  and  an  approximate  95%  LCB  computed  for  the  24  realizations 
of  the  sampling  plan  that  lead  to  lot  acceptance,  where  (i,j,k,l)  represent  the  number  of 
failures  in  tests  1-4  respectively 


(x,/,M) 

Rci 

95%  LCB 

(0000) 

1.000 

1.000 

(1000) 

0.989 

.900 

(0010) 

0.969 

.906 

(1010) 

0.969 

.355 

(0001) 

.950 

.850 

(0100) 

0.950 

.850 

(1001) 

0.950 

.323 

(1100) 

0.950 

.926 

(1011) 

0.944 

.302 

(1110) 

0.944 

.300 

(1111) 

0.939 

.765 

(0020) 

0.938 

.375 

(1020) 

0.937 

.313 

(1121) 

0.937 

.751 

(0011) 

0.920 

.323 

(0110) 

0.920 

.323 

(1101) 

0.920 

.775 

(1021) 

0.913 

.765 

(1120) 

0.913 

.761 

(0101) 

0.902 

.SCO 

(0021) 

0.391 

.775 

(0120) 

0.391 

.738 

(0111) 

0.374 

.750 

(0121) 

0.346 

.717 

have  passed  the  simple  environmental  test); 

•  x ,3  be  the  number  of  failures  resulting  from  both  the  manufacturing  and  the  environ- 
mental tests. 

The  likelihood  L  is  then  proportional  to 

L    oc     P(£,i)Tl(l-P(JE,i))"1-;Ej 

i 

H  P{FA  n  E,y-P{E}  n  E^y* P{E{  n  E,)T---P(F^ n  P,c)r^. 

(=2 

Let  n  =  5Z*_j  nt.  Conditioning  on  the  event  £"i  gives 

4 

J]  P(/:,|£'1)-(1  -  P(E,-|Ei))'«P(El-|Ef)*«(]  -  P(Ef-|Ef))*«. 
Therefore,  the  MLE's  of  interest  are 

Epl^r  +  J"i.l 


W)    = 


n 


P(Ei\Ei)    =  '       fori  =  2.3.-1. 

With  these  MLE's  in  hand,  we  ran  explain  the  behavior  of  the  estimates  in  the  condi- 
tional independence  case.  When  Eo.  £3,  £4  are  independent  given  E\.  we  have 


Rci    =    PiEOYlPiEilEr) 

(=2 

v-4  1  4 


i=2  *•■  +  *« 


When  the  Xj,i  are  not  observed,  the  maximum  likelihood  approach  assigns  values 

P(gfnfi) 

7  i.i    =    r*- 

P(P,n£",) 


Ri  -  Ri 

=     Xl~~R~ 

- 
to  j,.i  where  R{  are  found  by  maximizing  (1).    This  is  what  leads  to  the  seemingly  non- 
sensical ordering  of  reliabilities  seen  in  table  2.1.  The  worst  case  was  failure  configuration 
(1.  1.2. 1),  for  which  we  can  compute 


.ri  i  =  >i  i      =     0 


onf.  95  -.9375, 

•'Ml      =     -*0( — — ) 

.937o 


«     0.01333. 
Alternatively,  configuration  (0.1.2.1)  results  in 

v     .95     ; 

%      1 

1  -  .9375, 
v     .9375     ; 

~    2 

,  ^  -  -93  . 

SS  1 

Thus,  the  second  configuration,  although  being  clearly  superior  to  the  first,  results  in  assign- 
ing blame  to  the  environments  as  the  cause  of  failure.  Since  our  definition  of  reliability  is 
the  joint  probability  of  activating  under  all  circumstances,  the  failure  configurations  which 
appear  to  point  to  the  manufacturing  test  as  the  cause  of  most  of  the  failures  actually  yield 
greater  reliabilities. 
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Allowing  the  maximum  likelihood  procedure  to  assign  values  to  the  censored  observa- 
tions leads  to  an  ordering  of  the  estimates  of  R  that  are  counterintuitive.  In  the  next 
section,  we  will  develop  a  bound  for  R  which  doesn't,  use  conditional  independence,  and 
which  produces  an  ordering  of  estimates  of  R  which  is  more  believable. 

2.3     A  Reasonable  Lower  Bound 

An  alternative  approach  is  to  note  that  no  matter  what  the  censored  observations  are.  we 
still  have 

P(£i)     >     ^^ 
P(Ei\Ei)    >     -  i  =  2.3.4. 

Thus,  under  the  assumption  of  conditional  independence 


£C7>(£k£i)jj£i 

n  XX    r>  ■ 


Note  that  the  lower  bound  of  Rci- 


i 
•r, 


Au  =  (£k^)TXi!  (2) 

is  also  equal  to  the  MLE  when  there  is  as  much  dependence  between  E\ . . . ,  £4  as  is  possible. 
This  maximum  dependence  case  arises  if  we  assume  that  all  of  the  failures  observed  are  the 
result  of  BOTH  the  manfacturors  test  and  and  environmental  test.  i.e.  .r,,3  =  /?,■  —  ,rt,  i  = 
2.3,4. 

RlB  for  the  failure  configurations  leading  to  lot  acceptance  are  shown  in  Table  3.  There 
are  two  advantages  to  using  Rib  rather  than  Rci  to  estimate  R.  First,  estimates  Rib  are 
ordered  in  a  reasonable  way.   failure  configurations  with  more  failures  always  have  lower 
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Table  3:  The  lower  bound  for  the  MLE  of  R  computed  for  the  24  realizations  of  the  sampling 
plan  leading  to  lot.  acceptance  where  (i,j,k,l)  represent  the  number  of  failures  in  tests  1-4 
respectively. 


(0000) 

1.000 

(1000) 

0.939 

(0010) 

0.953 

(1010) 

0.943 

(0001) 

0.940 

(0100) 

0.940 

(1001) 

0.933 

(1100) 

0.930 

(0020) 

0.917 

(1020) 

0.907 

(0011) 

0.900 

(0110) 

0.900 

(1011) 

0.390 

(1110) 

0.390 

(0101) 

0.333 

(1101) 

0.373 

(0120) 

0.362 

(0021) 

0.362 

(1021) 

0.352 

(1120) 

0.352 

(0111) 

0.346 

(1111) 

0.336 

(0121) 

0.309 

(1121) 

0.300 
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values  of  Rlb-  Second.  Rlb  provides  a  lower  bound  for  R  under  much  weaker  assumptions 
about  the  dependence  of  the  test  outcomes. 

Following  [4],  suppose  that  a  device  is  more  likely  to  activate  after  exposure  to  an 
environment  if  it  has  activated  after  exposure  to  a  previous  environment.  Specifically, 
suppose 

PiEslEiDEi)     >     P(£3|£i) 

P( £4 ! Ei  n  £2  n  £3)    >    P{EA\E{). 

then 

R    fe    P{El)P(E-2\El)P(E:i\El  n  £2)£(£4!£i  n  E2  n  £3) 

>     £(£1)JP(£,|£1)£(£3!£i)£(£4!£i). 

Thus.  /?£s  given  in  (2)  provides  a  lower  bound  for  the  MLE  of  R  under  these  much  more 
plausible  assumptions. 

2.4     Lower  Confidence  Bounds 

Also  given  in  Table  2  are  approximate  95%  lower  confidence  bounds  found  by  bootstrap- 
ping [3].  For  each  of  the  24  realizations  under  consideration.  1000  bootstrap  samples  were 
generated  as  follows.  Let,  r,-.fc,  i  =  1 4,k  =  1 .....  24  represent  the  value  of  /?,  comput- 
ed for  the  k  realization.  Then,  for  the  k  realization,  the  bootstrap  samples  consist  of 
generating  1000  independent  realizations  of  X\^  . . . ,  X^k  where  X^k X^j.  are  indepen- 
dent and  Xuk  ~  B inomial(nt,  r,,A.).  From  each  of  the  1000  bootstrap  samples,  the  MLE  of 
R  is  computed,  and  the  9o9c  lower  confidence  bound  is  taken  as  the  5fA  percentile  of  the 
empirical  distribution  of  these  estimates  of  R. 


The  lower  confidence  bound  for  configuration  (1, 1,  2. 1)  is  slightly  higher  than  the  lower 
confidence  bound  of  [4].  These  lower  confidence  bounds  are.  however,  approximations  and 
are  possibly  biased.  To  see  how  these  lower  confidence  bounds  perform,  it  is  a  simple 
matter  to  compute  the  probabilities  that  they  cover  the  true  value  of  R  under  prescribed 
dependence  assumptions.  Returning  to  the  conditional  independence  case,  we  can  use  (1)  to 
compute  the  probability  of  any  sampling  outcome  given  values  of  R\  . .  . ,  R±.  For  simplicity, 
take  ./?•,  =  ..  .  =  /?4.  so  that,  under  conditional  independence.  R  =  R,.  For  R  >  0.9.  the 
confidence  intervals  given  by  the  lower  confidence  bounds  in  Table  2  ALWAYS  cover  R  if 
there  is  at  least  one  failure.  Thus,  the  true  level  of  confidence  for  these  bounds  is  precisely 
the  probability  that  at  least  one  failure  occurs  during  lot  acceptance  testing.  1  —  o  =  1  —  R92. 
(n  —  92  is  the  number  of  tested  units.)  Thus,  for  R  —  0.99.  the  confidence  level  for  the 
lower  bound  is  approximately  60%.  and  decreases  to  37%  when  R  increases  to  0.995. 

This  strange  effect  is  due  to  the  design  of  the  lot  acceptance  plan  .  The  plan  is  designed 

to  ensure  with  959c  confidence  that  a  lot  is  accepted  if  niin(Ri R+)  >  0.99.  With  higher 

reliabilities,  there  is  a  very  good  chance  that  there  won't  be  any  failures  during  the  test, 
and  we  are  powerless  to  produce  an  accurate  95%  lower  confidence  bound  for  R. 

In  this  section,  we  have  attempted  to  produce  an  accurate  point  estimate  and  lower 
confidence  bound  for  R.  with  the  ultimate  goal  of  establishing  thresholds  for  assigning 
bonuses.  In  the  next  section,  we  reapproach  the  problem  of  assigning  bonuses. 

3      Alternative  Formulation 

In  this  section,  we  take  a  fresh  approach  to  the  problem  of  awarding  bonuses.  Our  decision 
will  be  of  the  form,  award  a  bonus  if 


1  1 


R(X)  >  c 

where  R(X)  is  a,  statistic,  possibly  an  estimate  of  reliablility,  based  on  the  number  of 
devices  that  activate  in  tests  1-4,  X  =  (X\, ....  Xj).  As  we  have  stipulated  in  the  previous 
section,  for  R(X)  to  be  reasonable,  it  must  be  nondecreasing  in  each  of  its  arguements,  thus 

if  %i  >  Vi>.  i  —  1. ...  ,-1  then  R(x)  >  R(y)  where  x  =  (x^ x+)  and  y  =  (i/i y±).  In 

the  previous  section.  Rib  has  this  property. 

The  ultimate  goal  of  the  bonus  program  is  to  reward  manufacturers  who  provide  goods 
which  are  more  useful  in  operations  than  the  government-specified  level  of  acceptability. 
Thus,  the  bonus  rule  will  be  based  on  the  goal  of  rewarding  all  manufacturers  for  which 

min(R\ #4)  >  r  with  probability  at.  least  1  —  o.  The  decision  rule  which  accomplishes 

this  is  to  award  bonuses  if 

R{X)  >  c* 

where  c*  is  the  largest  value  of  r.  for  which 

1  -  a  <  P(R(X)  >  c)  (3) 

for  R1=...  =  R4  =  r. 

As  an  example,  take  R(X)  to  assign  numbers  1. . . . ,  2-4  to  the  outcomes  leading  to  lot 
acceptance  as  listed  in  Table  4.  Table  4  gives  the  cumulative  probabilities  F(/?(X)  >  c) 
when  R\  =  .  . .  =  /?4  =  r  for  different  values  of  r.  Thus,  if  bonuses  are  to  be  award- 
ed to  manufacturers  for  which   min{R\ R\)   >  0.995  with  95%  chance  of  identifying 

these  vendors  based  on  a  lot  acceptance  sampling  plan,  then  the  threshold  score  c*  =  17. 
Here,  bonuses  are  given  to  manufacturers  which  have  one  or  zero  total  failures,  or  have 
(0.  0. 1. 1).  (0, 1,  0. 1).  or  (0, 1. 1.  0)  failures.  Note  that  in  this  example  the  values  were  arbi- 
trarily assigned  and  don't  take  into  account  the  fact  that  test  3  uses  a  larger  sample  of  32 
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while  the  other  tests  are  based  on  a  sample  size  of  20. 

Depending  on  the  environment,  that  the  device  will  be  used  in.  the  reliability  is  prob- 
ably closer  to  a  weighted  average  of  R\. . . . ,  R\.  Because  these  weights  differ  for  differ- 
ent types  of  pyrotechnic  devices  and  the  weights  themselves  must  be  estimated  based  on 
the  mission  the  devices  are  used  in.  a  simple  and  conservative  alternative  is  to  consider 

Rmtn  =   mm(Ri R4).     The  minimum  also  has  the  attractive  feature  that  it's  MLE 

can  be  found  from  the  lot  acceptance  data  without  assuming  anything  about  the  depen- 
dencies between  E\ E\.    Table  5  gives  the  MLE  R,mn  =  min(R\ .  R4)  along  with 

P(Rmm  >  c)  when  R\  =  . . .  =  R4  =  r  for  various  values  of  r. 

To  see  that  this  rule  acheives  the  desired  goal,  in  otherwords  that 

1  -o  <  P(R(X)  >  c*) 

for  Rmin  >   r  where  c*  is  chosen  as  in  (3).  and  /?(X)  is  a  nondecreasing  function  of 

.Yj ,Y4,  we  need  the  notion  of  stochastic  ordering.   The  random  variable  X  is  stochas- 

ticaily  greater  than  the  random  variable  Y.  denoted  .Y  >  Y.  if  P(X  >  x)  >  P(Y  >  x)  for 
all  x.  In  the  multivariate  extension  of  this  notion,  the  random  vector  X  is  stochastically 
greater  than  the  random  vector  Y  if 

E[h(X)\  >  E[h(Y)} 

for  all  real- valued  functions  h  which  are  nondecreasing  in  their  arguements  for  which 
the  expectation  exists. 

Now  let  X\ X±  be  independent  with  X,  ~~  BiTtomial(nt,  Rt)  and   YJ VJ  be  in- 

st 

dependent  with   V,'  ~  B inomial(nt.  r).  where  R,  >  r.  Then  X,  >  Yt  for  1  =  1 4  and  by 

Theorem  4.13  of  [1] 

X  =  (,Y, Y,)>  Y  =  (>i JS). 

16 


Table  4:    Cumulative  probabilities  for  R(X)  when  R\  =  r,  ...,itt  =  r,  where  (i,j,k,l) 
represent  the  number  of  failures  in  tests  1-4  respectively. 

P{R{X)  >  c) 

(ij,U)  R-.99           R=».995            R=».999 

24  (0000)  0.401 

23  (0001)  0.432 

22  (0010)  0.510 

21  (0100)  0.691 

20  (1000)  0.772 

19  (0011)  0.793 

13  (0101)  0.314 

17  (0110)  0.340 

15  (1001)  0.356 

15  (1010)  -0.332 

14-  (1100)  0.399 

13  (0020)  0.919 

12  (0111)  0.924 

11  (1011)  0.929 

10  (1101)  0.932 

9  (1110)  0.933 

3  (0021)  0.942 

7  (0121)  0.942 

5  (1020)  0.947 

5  (1111)  0.943 

4-  (0121)  0.943 

3  (1021)  0.950 

2  (1120)  0.950 

1  (1121)  0.950 


0.534 

0.913 

0.697 

0.9*31 

0.300 

0.960 

0.363 

0.979 

0.926 

0.997 

0.936 

0.993 

0.943 

0.993 

0.953 

0.999 

0.959 

1.000 

0.970 

1.000 

0.976 

1.000 

0.934 

1.000 

0.935 

1.000 

0.936 

1.000 

0.937 

1.000 

0.933 

1.000 

0.939 

1.000 

0.933 

1.000 

0.939 

1.000 

0.939 

1.000 

0.939 

1.000 

0.990 

1.000 

0.990 

1.000 

0.990 

1.000 
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Table  5:    Cumulative  probabilities  for  -Rm,n  when  R\  =  r,  ...R4  =  r,  where  (i,j,k,l) 
represent,  the  number  of  failures  in  tests  1-4  respectively. 


1.000 
0-.939 
0.969 

0.960 
0.950 


0.933 


(0000) 
(1000) 
(0010) 
(1010) 
(0110) 
(0001) 
(0100) 
(0011) 
(0101) 
(1001) 
(1100) 
(0111) 
(1011) 
(1101) 
(1110) 
(0020) 
(0021) 
(0121) 
(1020) 

(1111) 
(0121) 
(1021) 
(1121) 
(1121) 


P(Rmm  >  C) 

R=».99 

R-.995 

R-.99 

0.401 

0.634 

0.913 

0.432 

0.697 

0.931 

0.636 

0.309 

0.961 

0.667 

0.319 

0.962 

0.913 

0.930 

1.000 

0.952 


0.990 


1.000 


0.950 


0.990 


1.000 
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This  gives  us  R(X)  >  R(Y)  so  that 

P(R(X)  >  O  >  P(R{Y)  >c*)>l-a. 

More  formally,  the  procedure  to  award  bonuses  tests  the  null  hypothesis  H0  :  R[  > 
r, . . . ,  /?4  >  r.  Defining  the  null  hypothesis  in  this  way  alows  us  to  control  the  level  of 
significance  o.  i.e.  the  probability  of  not  awarding  a  bonus  when  one  is  deserved.  The 
likelihood  ratio  test  (e.g.  [2])  for  this  problem  is  to  award  a  bonus  (i.e.  fail  to  reject  H0)  if 
i?(X)  >  c*  where  R(X)  is  given  by 

-  y>7»(i)  +  (nt  -  Xi)ln(±^)]I(Ri  <  r) 
Hi  1  -  H{ 

where  J  {A)  is  the  indicator  function  of  the  set  A. 

4     Conclusion 

There  are  several  factors  that  make  using  acceptance  test  data  to  award  bonuses  challenging. 
They  all  stem  from  the  fact  that  the  data  is  gathered  from  a  lot  acceptance  sampling  plan 
which  was  not.  designed  to  estimate  an  overall  reliability,  nor  was  it  designed  to  give  good 
estimates  of  reliabilities  when  they  are  higher  than  the  minimal  lot  acceptance  standards. 

To  use  the  data  from  the  lot  acceptance  sampling  plan,  the  most  reasonable  approach 
seems  to  use  /?mm,  and  its  distribution  for  /?i  =  . . .  =  R±  to  decide  whether  to  award  a 
bonus. 

There  are  several  ways  that  this  decision  rule  can  be  improved.  These  initiatives  were 
mentioned  earlier,  and  include 

1.   Take  larger  samples  or  sample  sequentially.  This  will  reduce  the  chances  of  awarding 
a  bonus  to  underserving  manufacturers. 
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2.  Construct  a  better  measure  of  overall  reliability.  A  viable  option  is  a  weighted  average 

R  =  Y.  u:R. 

,-, 

where  the  weights  ?e,  represent  the  proportion  of  the  rieviees  in  the  field  that,  are 
subjected  to  environment  i.  This  would  involve  either  constructing  or  estimating 
different  weights  for  different  types  of  devices. 

3.  Use  R  =  P(E\  H  Ei  H  £"3  fl  £"4)  as  the  measure  of  overall  reliability,  but  collect  data  to 
estimate  the  interaction  between  the  environmental  tests.  R  could  then  be  estimated 
using  a  loglinear  model  based  on  censored  observations. 
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